Field of the Invention
The present invention relates to an image processing apparatus, and more particularly to a technique to extract a sky region using a depth image and a color image.
Description of the Related Art
To reproduce memory colors in an imaging apparatus or an image processing apparatus, techniques to extract sky, human skin, the green of trees or the like as a region of interest, has been proposed. In these techniques, the region is normally determined using hue, chroma and brightness.
In recent years, an imaging apparatus configured to acquire a depth image at the same time with acquiring a color image is widely used, and an object extraction method using the depth image has been proposed. Various methods to measure depth have been proposed, such as an active method represented by a time of flight (TOF) method, and a passive method, such as a stereo method and a depth from defocus (DFD) method. In any of these methods, the range where depth can be measured is limited, so it is difficult to detect a sky region and measure the shape thereof correctly and at high resolution.
As a consequence, accurately extracting a sky region is important not only for reproducing memory colors of a color image, but also for correcting the depth of the depth image.
In Japanese Patent No. 4040625 (PTL1), in order to prevent correction of a memory color which should not be corrected, the correction intensity is changed depending on the color even if the color has the same correction target hue, so as to make the correction look visually natural. Further, whether a sky is included in the image or not is determined using the hue in the image and the photographing conditions, so that a region other than the sky will not be corrected by mistake.
In Japanese Patent Application Laid-Open No. 2013-182330 (PTL2), pixels existing in a specific depth range are selected using the depth information for extracting an object, and the object region is determined by calculating the degree of interest based on the degree of change of the pixels.    PTL1: Japanese Patent No. 4040625    PTL2: Japanese Patent Application Laid-Open No. 2013-182330